In this so-called information age, transformation of data, i.e., the modification and/or re-arrangement of data from one form into another, has become a ubiquitous task. While many data transformation tasks are relatively straightforward, many others are quite complex and carry significant consequences for failure to correctly transform data as designed. For example, in the pharmaceutical and/or medical device industries, clinical trials are conducted to facilitate the collection of significant quantities of safety and efficacy data for new drugs or devices.
Depending on the type of product and the stage of its development, clinical trials typically enroll healthy volunteers and/or patients into small studies initially, followed by larger scale studies in patients that often compare the new product with the generally accepted, standard course of treatment, i.e., treatment based on currently available pharmaceuticals or devices, if any. Generally, as positive safety data is gathered, the number of patients is typically increased during larger efficacy trials. Regardless of the type and size of a given clinical trial, the data obtained during the clinical trial must to be submitted to the responsible governmental regulatory agency to conduct a thorough review of the new product being developed. For example, in the United States, the Food and Drug Administration (FDA) is responsible for the approval of new drugs and medical devices.
The Clinical Data Interchange Standards Consortium (CDISC) has focused considerable effort on developing standards to help FDA in its review and approval process of safety and efficacy data. This standard format is sometimes referred to as the Study Data Tabulation Model (SDTM) format. There is an increasing demand for transforming data captured during clinical trials (which data can vary widely in its form and content) into the desired SDTM format. Typically, data transformation is divided into two steps: first, data mapping maps data elements from the source to the destination and captures any transformations that must occur and, second, code generation is performed to create the necessary transformation program, i.e., an executable software program that can be run on a computer system. In a typical transformation process, a technical design document, e.g., a metadata-based mapping sheet that specifies how to map input data to output data in accordance with a particular SDTM variable, is created to establish the necessary mapping rules. For example, a technical design document is often captured in the form of a spreadsheet in which individual rows set forth the desired data transformations. Thereafter, a data transformation tool (such as the “TableTrans” visual database programming software by CSS Informatics or the “SAS” Data Integration Studio visual design tool by SAS Institute Inc.) is employed to begin the design of the transformation program. Such design transformation tools employ a graphical user interface (GUI) in which icons representative of various transformational operations may be arranged and ordered as needed in accordance with the transformation rules described in the technical design document. Each icon represents relatively sophisticated data processing functions written in an underlying statistical or database programming language. Because the programs underlying each icon are fully tested, reliable data transformation programs can be devised. That is, by interpreting the arrangements and ordering of the icons established via the GUI, more complex data transformation programs may be generated based on the pre-built functions.
It is not uncommon for a single clinical trial to require 15-20 different data transformation programs created as described above. Given this, the entire process of developing a new study typically requires at least 15-20 days, assuming 3-4 resources working on the transformation programs as described above. Despite the use of the data transformation tools, the overall data mapping process currently remains a tedious process that is sometimes prone to error in capturing the requirements from the technical design document.
While the clinical trial example described above illuminates some of the shortcomings of the prior art, it is understood that these limitations are not the exclusive domain of data transformations employed in clinical trials. Indeed, virtually any endeavor requiring relatively complex data transformations, e.g., data analysis in financial transactions, would suffer from the same shortcomings. Thus, it would be advantageous to provide data transformation techniques, data transformation devices and systems that overcome the limitations of prior art techniques.